User interface widget modeling and placement

ABSTRACT

Techniques to facilitate visualization of an application associated with an industrial automation environment are disclosed. In at least one implementation, a display composer interface is presented that enables a user to design a customized display layout for the application associated with the industrial automation environment. Data display instructions are received comprising a user selection of at least one data item associated with an operation of a machine in the industrial automation environment. Position information that identifies where to display the at least one data item is also received. The data display instructions and the position information are processed to generate the customized display layout for the application. Based on the customized display layout, a graphical user interface is rendered to the application having the at least one data item positioned according to the position information.

RELATED APPLICATIONS

This application is a continuation of, and claims the benefit ofpriority to, U.S. Pat. No. 10,509,394, granted on Dec. 17, 2019, andentitled “USER INTERFACE WIDGET MODELING AND PLACEMENT,” which itselfclaims the benefit of, and priority to, U.S. Provisional PatentApplication No. 62/255,611, also entitled “USER INTERFACE WIDGETMODELING AND PLACEMENT”, and filed on Nov. 16, 2015, both of which arehereby incorporated by reference in their entirety for all purposes.

TECHNICAL FIELD

Aspects of the disclosure are related to computing hardware and softwaretechnology, and in particular to industrial automation applications.

TECHNICAL BACKGROUND

Industrial automation environments utilize machines during theindustrial manufacturing process. These machines typically have variousmoving parts and other components that continually produce operationaldata over time, such as pressure, temperature, speed, and other metrics.Reviewing and monitoring this operational data is of high importance tothose involved in operating the industrial automation environment.

In some cases, the operational data may be provided for display on avariety of systems and devices, such as notebook or tablet computersrunning standalone applications, web browsers, and the like, in order topresent the information to the user. For example, a tablet computercould receive a constant stream of the operational data over a networkand display this information dynamically, such as by providing livetiles, icons, charts, trends, and other graphical representations whichcontinually receive updated operational data from external data sources,which are typically located in the industrial automation environment.

Overview

Provided herein are systems, methods, and software to facilitatevisualization of an application associated with an industrial automationenvironment. In at least one implementation, a display composerinterface is presented that enables a user to design a customizeddisplay layout for the application associated with the industrialautomation environment. Data display instructions are receivedcomprising a user selection of at least one data item associated with anoperation of a machine in the industrial automation environment.Position information that identifies where to display the at least onedata item is also received. The data display instructions and theposition information are processed to generate the customized displaylayout for the application. Based on the customized display layout, agraphical user interface is rendered to the application having the atleast one data item positioned according to the position information.

This Overview is provided to introduce a selection of concepts in asimplified form that are further described below in the TechnicalDisclosure. It should be understood that this Overview is not intendedto identify key features or essential features of the claimed subjectmatter, nor is it intended to be used to limit the scope of the claimedsubject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. While several implementations are describedin connection with these drawings, the disclosure is not limited to theimplementations disclosed herein. On the contrary, the intent is tocover all alternatives, modifications, and equivalents.

FIG. 1A is a block diagram that illustrates an industrial automationenvironment in an exemplary implementation.

FIG. 1B is a flow diagram that illustrates an operation of a computingsystem in an exemplary implementation.

FIGS. 2-11D are block diagrams that illustrate various operationalscenarios involving a computing system in an industrial automationenvironment in exemplary implementations.

FIG. 12 is a block diagram that illustrates a computing system in anexemplary implementation.

DETAILED DESCRIPTION

The following description and associated figures teach the best mode ofthe invention. For the purpose of teaching inventive principles, someconventional aspects of the best mode may be simplified or omitted. Thefollowing claims specify the scope of the invention. Note that someaspects of the best mode may not fall within the scope of the inventionas specified by the claims. Thus, those skilled in the art willappreciate variations from the best mode that fall within the scope ofthe invention. Those skilled in the art will appreciate that thefeatures described below can be combined in various ways to formmultiple variations of the invention. As a result, the invention is notlimited to the specific examples described below, but only by the claimsand their equivalents.

Implementations disclosed herein provide for improved visualizations ofinformation items associated with applications executing on a computingsystem. In one implementation, an application associated with anindustrial automation environment may display a graphical user interfaceshowing one or more key performance indicators (KPIs) related toindustrial operations. For example, the graphical user interface coulddisplay KPIs related to operational data associated with machines in theindustrial automation environment, such as parameters, pressures,temperatures, speeds, production rates, or some other status metrics. Insome examples, the operational data for some KPIs may comprise dynamiccharts or trends, real-time video, or some other graphical content.

By interacting with the graphical user interface, a user may utilize adisplay composer to select one or more of the KPIs for display by theapplication, along with position information that identifies where onthe screen to display each of the selected KPIs. In someimplementations, the user can also define the size, shape, display style(i.e., gauge, trend, chart, indicator, or some other display type), andother properties of each KPI that is selected for display. In thismanner, the user is able to create a customized display screen thatpresents the selected manufacturing data in the arrangement and styledesired by the user. This custom display layout can be saved so that thevisualization of the data as defined by the user may be presented at alater time. When viewing the user-defined display, the application mayprovide a “live feed” of dynamic KPI data for display according to thearrangement, style, and other properties defined by the user. This feedof dynamic KPI data may be continuously updated by the application sothat the visualization of the KPI displayed shows the most currentinformation associated with the KPI.

Referring now to the drawings, FIG. 1A illustrates an exemplaryindustrial automation environment that includes a computing system thatmay be used to execute an application and its visualization process, andFIG. 1B illustrates an operation of a computing system in an exemplaryimplementation. FIGS. 2-11D illustrate several operational scenariosinvolving a computing system according to various exemplaryimplementations. FIG. 12 illustrates an exemplary computing system thatmay be used to perform any of the visualization processes andoperational scenarios described herein.

Turning now to FIG. 1A, a block diagram that illustrates an industrialautomation environment 100 in an exemplary implementation is shown.Industrial automation environment 100 provides an example of anindustrial automation environment that may be utilized to implement thevisualization processes disclosed herein, but other environments couldalso be used. Industrial automation environment 100 includes computingsystem 110, machine system 120, industrial controller 125, databasesystem 130, and application integration platform 135. Machine system 120and controller 125 are in communication over a communication link,controller 125 and database system 130 communicate over a communicationlink, database system 130 and application integration platform 135communicate over a communication link, and application integrationplatform 135 and computing system 110 are in communication over acommunication link. Note that there would typically be many more machinesystems in most industrial automation environments, but the number ofmachine systems shown in FIG. 1A has been restricted for clarity.

Industrial automation environment 100 comprises an automobilemanufacturing factory, food processing plant, oil drilling operation,microprocessor fabrication facility, or some other type of industrialenterprise. Machine system 120 could comprise a sensor, drive, pump,filter, drill, motor, robot, fabrication machinery, mill, printer, orany other industrial automation equipment, including their associatedcontrol systems. A control system comprises, for example, industrialcontroller 125, which could include automation controllers, programmablelogic controllers (PLCs), or any other controllers used in automationcontrol. Additionally, machine system 120 could comprise otherindustrial equipment, such as a brew kettle in a brewery, a reserve ofcoal or other resources, or any other element that may reside in anindustrial automation environment 100.

Machine system 120 continually produces operational data over time. Theoperational data indicates the current status of machine system 120,such as parameters, pressure, temperature, speed, energy usage,operational equipment effectiveness (OEE), mean time between failure(MTBF), mean time to repair (MTTR), voltage, throughput volumes, times,tank levels, or any other performance status metrics. The operationaldata may comprise dynamic charts or trends, real-time video, or someother graphical content. Machine system 120 and/or controller 125 iscapable of transferring the operational data over a communication linkto database system 130, application integration platform 135, andcomputing system 110, typically via a communication network. Databasesystem 130 could comprise a disk, tape, integrated circuit, server, orsome other memory device. Database system 130 may reside in a singledevice or may be distributed among multiple memory devices.

Application integration platform 135 comprises a processing system and acommunication transceiver. Application integration platform 135 may alsoinclude other components such as a router, server, data storage system,and power supply. Application integration platform 135 may reside in asingle device or may be distributed across multiple devices. Applicationintegration platform 135 may be a discrete system or may be integratedwithin other systems—including other systems within communication system100. In some examples, application integration platform 135 couldcomprise a FactoryTalk® VantagePoint server system provided by RockwellAutomation, Inc.

Computing system 110, referred to in FIGS. 1-12, may be representativeof any computing apparatus, system, or systems on which thevisualization processes disclosed herein or variations thereof may besuitably implemented. Computing system 110 provides an example of acomputing system that could be used as a either a server or a clientdevice in some implementations, although such devices could havealternative configurations. Examples of computing system 110 includemobile computing devices, such as cell phones, tablet computers, laptopcomputers, notebook computers, and gaming devices, as well as any othertype of mobile computing devices and any combination or variationthereof. Examples of computing system 110 also include desktopcomputers, server computers, and virtual machines, as well as any othertype of computing system, variation, or combination thereof.

Computing system 110 includes processing system 101, storage system 103,software 105, communication interface 108, and user interface 109.Processing system 101 is operatively coupled with storage system 103,communication interface 108, and user interface 109. Processing system101 loads and executes software 105 from storage system 103. Software105 includes a visualization process. When executed by computing system110 in general, and processing system 101 in particular, software 105directs computing system 110 to operate as described herein for avisualization process or variations thereof. Computing system 110 mayoptionally include additional devices, features, or functionality notdiscussed here for purposes of brevity. An exemplary operation ofcomputing system 110 to facilitate visualization of an applicationassociated with an industrial automation environment will now bediscussed with respect to FIG. 1B.

FIG. 1B is a flow diagram that illustrates an operation 200 in anexemplary implementation. The operation 200 shown in FIG. 1B may also bereferred to as visualization process 200 herein. The steps of theoperation are indicated below parenthetically. The following discussionof operation 200 will proceed with reference to computing system 110 andmachine system 120 of FIG. 1A in order to illustrate its operations, butnote that the details provided in FIG. 1A are merely exemplary and notintended to limit the scope of process 200 to the specificimplementation shown in FIG. 1A.

Operation 200 may be employed to operate a computing system tofacilitate visualization of an application associated with an industrialautomation environment, such as industrial automation environment 100 ofFIG. 1A. Typically, operation 200 may be performed by computing system110, although operation 200 could be executed by any system or device insome implementations. As shown in the operational flow of process 200,computing system 101 presents a display composer interface that enablesa user to design a customized display layout for an applicationassociated with an industrial automation environment (201). In someexamples, the display composer interface provides a visualization layouteditor for the user to customize the display of an industrial automationapplication. For example, the user may utilize the display composerinterface to select desired data items for viewing and the displaytypes/styles for presenting that data, along with the arrangement of thevarious data items and their display types in order to create acustomized display layout. In at least one implementation, the displaycomposer interface provides at least one selection menu and a displaycanvas comprising a grid for designing the customized display layout.For example, the display composer interface could provide both a modeltoolbox panel menu for selecting various data items and a widget toolboxpanel menu for selecting different data display types, such as valueboxes, indicators, gauges, and the like. The user can then either drag aparticular data display type from the widget toolbox panel to thedisplay canvas to create a display item of a particular type, and thenassign a specific data item to that display type, or the user can drag adata item from the model toolbox panel menu to the canvas, in which casea default display type will be created for the selected data item and towhich the data item will automatically be assigned. Typically, duringthe drag of either a data display type from the widget toolbox panel ora data item from the model toolbox panel over the grid on the displaycanvas, a placeholder will appear on the grid to indicate the positionwhere the widget will be inserted after the drop. Other implementationsand functionality of the display composer interface are possible andwithin the scope of this disclosure.

After presenting the display composer interface, computing system 110receives data display instructions comprising a user selection of atleast one data item associated with an operation of a machine in theindustrial automation environment (202). For example, in at least oneimplementation, the user selection of the at least one data item couldcomprise one or more key performance indicators (KPIs) related toindustrial operations, such as KPIs related to operational dataassociated with machine system 120 of industrial automation environment100, such as parameters, pressures, temperatures, speeds, productionrates, or some other status metrics. Each data item selected by the userin the data display instructions typically includes an associated datadisplay type, such as indicators, value boxes, gauges, plots, trends,charts, grids, visualizations of vessels such as tanks or vats, or anyother data visualization types. In some examples, the data display typesfor some data items may comprise dynamic charts or trends, real-timevideo, or some other live graphical content. Further, in someimplementations, the user can also define the size, shape, display style(i.e., gauge, trend, chart, indicator, value box, or some other displaytype), and other properties of each data item that is selected fordisplay in the data display instructions. For example, the user couldselect the desired data display type when providing the data displayinstructions, such that receiving the data display instructionscomprises receiving a data display type for displaying the at least onedata item. Further, in some implementations, receiving the data displayinstructions could comprise receiving display size instructions for theat least one data item. For example, the display size instructions couldindicate the size of an associated data display type for each of thedata items selected by the user in the data display instructions. Inaddition to the above, other customized properties and layout designoptions may be included in the data display instructions are within thescope of this disclosure.

Computing system 110 also receives position information that identifieswhere to display the at least one data item (203). Typically, to providethe position information, the user drags a data item or a data displaytype from a selection menu to a display canvas area of the displaycomposer interface and positions the data item as desired. In at leastone implementation, the display canvas comprises a grid for designingthe customized display layout, which provides the user with a placementguide for positioning the various data items and/or data display types.The position information typically includes the desired displaylocations for one or more of the data items and/or data display types asindicated by the user.

Computing system 110 processes the data display instructions and theposition information to generate the customized display layout for theapplication (204). Typically, computing system 110 is able to generatethe customized display layout for the application based on the variousdata items and/or data display types indicated in the data displayinstructions and arranged on the display screen as indicated in theposition information. As discussed above, the data display instructionscould also include other customized display information such as sizes,shapes, styles, and other properties attributed to the data items andtheir associated data display types. Further, in at least oneimplementation, processing the data display instructions and theposition information to generate the customized display layout for theapplication could comprise altering the position information thatidentifies where to display the at least one data item to avoidoverlapping with other data items that were previously positioned in thecustomized display layout. For example, if a user initially places adata item and/or its corresponding data display type at a specificposition in the grid, and subsequently drags another data display typeover the location on the grid occupied by the display type of thepreviously-placed data item, this avoiding mechanism will relocate thedisplay type of the initial data item that the newly placed display typeis colliding with to make room for the display type of the new dataitem. From the user's perspective, this avoiding mechanic will typicallyoperate while the user is providing the data display instructions andthe position information while designing the customized display layoutfor the application, and will also operate when data display types areresized and the resulting larger size causes the resized display type tooverlap another data display type that was already positioned on thecanvas grid.

Based on the customized display layout, computing system 110 renders agraphical user interface to the application having the at least one dataitem positioned according to the position information (205). Typically,after designing the customized display layout, when the user launchesthe application, computing system 110 renders a graphical user interfaceto the application based on the customized display layout, which resultsin the at least one data item positioned according to the positioninformation. The desired data items are thus displayed to the user inthe selected data display types and arranged according to the user'sposition instructions, which provides the user with the most optimizedand beneficial interface to the application for that particular user.

Advantageously, the above techniques enable a user of an industrialautomation application to design a customized display layout that may beused to render a graphical user interface to the application accordingto the user's specifications. By providing the data display instructionsand the position information using the display composer interface, theuser is able to fully customize the display of information whensubsequently executing the industrial automation application. In thismanner, the techniques described herein provide the technical advantageof electronically storing customized display layouts of industrialinformation that may be used to render different graphical userinterfaces to an industrial automation application for a multitude ofusers. Further, by providing only the data items and data display typesselected by each individual user when designing their customized displaylayouts, the load on the processors, mechanical components, and otherelements of an industrial automation environment may be reduced,resulting in significant energy savings by avoiding unnecessaryoperations. In this manner, a user of an application in an industrialautomation environment is presented with only the data items selected bythe user contained in data display types arranged according to theuser's specifications, thereby optimizing the utility of the applicationfor the user and greatly enhancing the user experience.

Several illustrative examples of various possible implementations of theabove visualization techniques will now be discussed. The followingexamples as illustrated in FIGS. 2 through 11D could be executed bycomputing system 110 and other elements of industrial automationenvironment 100, and could also be combined with operation 200 of FIG.1B in some implementations.

FIG. 2 is a block diagram that illustrates an operational scenarioinvolving computing system 110 in an exemplary implementation. As shownin FIG. 1A, computing system 110 includes user interface 109, which inturn comprises display system 111. In this example, user interface 109and display system 111 comprise a touch screen that displays a graphicaluser interface for the user to interact with and accepts input commandsfrom the user via the user's touch on the surface of the touch screen.However, user input could be provided by a variety of differenttechniques in other examples, such as entering text commands with akeyboard, clicking a mouse pointer, speaking a voice command, or anyother user input mechanism.

FIG. 2 illustrates two different display screens that may appear ondisplay system 111 of computing system 110. In the display screenappearing at the top of FIG. 2, the user is presented with a userinterface for an application associated with an industrial automationenvironment, such as the environment 100 illustrated in FIG. 1A. Asshown in the display screen appearing at the top of FIG. 2, the userinterface presents data associated with a machine named “Extuder001”. Atthe bottom left-hand corner of this display, the user can select anoption to open a new display composer interface, which in this exampleis designated with a “plus” icon, although any graphic, text, or otherindicator could be used to open a new display composer in otherexamples. To facilitate this discussion, a circle surrounds the “plus”icon in this example to highlight the “new display composer” option, butnote that the plus icon is not typically encircled on the application'suser interface. Also, the user can open the composer at any level of themodel hierarchy. Once the user selects the “plus” icon to open thedisplay composer, the user is presented with the display composerinterface, as shown in the display screen appearing at the bottom ofFIG. 2. The details of the display composer interface will now bedescribed with respect to FIG. 3.

FIG. 3 is a block diagram that illustrates an operational scenarioinvolving computing system 110 in an exemplary implementation. In FIG.3, an exemplary interface that enables a user to compose and edit customdisplays is shown on the display screen of computing system 110, labeled“New Display”. In some examples, the user may access this displaycreation interface by selecting an option to create a new display asdiscussed above with respect to FIG. 2.

In this example, the display composer interface includes two sections,divided into a menu on the left-hand side and a display composer on theright, although these windows could appear anywhere on the screen inother examples. The menu in the left-hand pane provides various modelsand widgets that provide visualizations of operational data items, andthe composer section in the right-hand pane provides a blank canvas thatmay be used to arrange the data items as the user desires them to bepresented. In this example, the display composer canvas comprises avisualization of a grid onto which the user can insert and positionvarious widgets that display operational data. The gridlines displayedin the composer canvas assist the user in positioning the variouswidgets and data items when designing a custom display.

As shown in the menu on left-hand side of the display composer, the usercan select from either a list of widgets or a list of models, and theselists are selected by the tabs labeled “Model” and “Widgets” at thebottom of the left-hand side pane. In FIG. 2, the “Model” tab isselected, which provides various information and data values for aparticular machine or model. The heading at the top of the “Model” tabon left-hand side pane is labeled “Extruder001”, which indicates thatthe information listed below is associated with the “Extruder001” model.Under the “General” heading, various identifying information about themodel is presented, such as the fully qualified name (file name), modelname, and description. Under the “Tags” heading, various data values arepresented, such as whether or not the machine has received a startcommand, hydraulic pressure, stroke length, reservoir level, cutterposition, batch number, piece number, the number of defects per batch,and the like. Of course, the number and type of data values that areshown in FIG. 3 will differ depending on the type of model or machineselected; the ones shown in FIG. 3 are merely exemplary. The effect ofthe user selecting the “Widgets” tab at the bottom of the left-hand menuto bring up the widgets toolbox panel will now be discussed with respectto FIG. 4.

FIG. 4 provides another operational scenario involving computing system110 in an exemplary implementation. In this example, the user hasselected the tab labeled “Widgets” at the bottom of the left-hand sidemenu of the display composer interface, instead of the “Model” tab asshown in FIG. 3. Selecting the “Widgets” tab displays the widgetstoolbox panel which provides the user with several different displayitems. In particular, the user can select from an indicator, value box,gauge, linear gauge, Scalable Vector Graphics (SVG) image, edge image,tank, raster image, trend, trend report, XY plot report, map, ReportDefinition Language Client (RDLC) report, alarm grid, transaction grid,query, test control, and others. In some examples, additional types ofreports, charts, and other widgets and data types may also be shown whenthe “Widgets” tab is selected and may be available for the user toselect but are not shown in FIG. 4 for clarity.

To begin designing a customized display or dashboard, the user coulddrag one of the data types listed in the “Widgets” menu on the left-handside pane over to the right-hand side display canvas, causing a blankwidget of the type selected to appear in the display composer. The userthen typically assigns a particular data source to the blank widget,such as the pump speed of a particular machine or any other value. Theuser can further customize the display by resizing the widget, definingcustom display attributes and other properties, and positioning thewidget in a desired area of the display screen, among otherfunctionality. Examples of these and other features will now bedescribed with respect to FIGS. 5A-11D.

FIGS. 5A and 5B are block diagrams that illustrate an operationalscenario involving computing system 110 in an exemplary implementation.FIG. 5A depicts a display composer interface with a user adding a widgetto an empty display canvas. Specifically, the user has selected a “valuebox” widget from the widget toolbox panel and has dragged the value boxwidget into the right-hand display canvas grid. While the user isdragging the widget over the grid, a placeholder appears on the gridwhich indicates the position where the widget will be inserted after theuser “drops” the widget, as shown by the dotted line forming the boxthat appears below the words “Value Box” on the display canvas editor.FIG. 5B illustrates the effect of the user dropping the “Value Box”widget on the grid of the display canvas. After the user positions the“Value Box” widget as shown in FIG. 5A and selects to “drop” the widgetonto the display canvas grid, a blank “Value Box” widget appears on thecanvas grid as shown in FIG. 5B. The value box appearing in FIG. 5B hasquestion marks shown within it because the user has not yet assigned avalue to the value box. The user could assign any value for display inthe value box by clicking on the “gear” icon in the upper right-handcorner of the value box widget to access the widget's properties menu.An example of a user selecting a data item from the “Model” toolboxpanel to add a widget to a display design canvas will now be discussedwith respect to FIGS. 6A and 6B.

FIGS. 6A and 6B are block diagrams that illustrate an operationalscenario involving computing system 110 in an exemplary implementation.FIG. 6A depicts a display composer interface with a user adding a widgetto an empty display canvas selected from the “Model” toolbox panel.Specifically, the user has selected a “Start Command” data item from the“Tags” menu of the “Model” toolbox panel and has dragged the “StartCommand” into the right-hand display canvas grid. While the user isdragging the data tag over the grid, a placeholder appears on the gridwhich indicates the position where the widget for the data tag will beinserted after the user “drops” the tag, as shown by the dotted lineforming the box that appears below the words “Start Command” on thedisplay canvas editor of FIG. 6A. FIG. 6B illustrates the effect of theuser dropping the “Start Command” data tag on the grid of the displaycanvas. After the user positions the “Start Command” data tag as shownin FIG. 6A and selects to “drop” the tag onto the display canvas grid, adefault widget for the “Start Command” data type appears on the canvasgrid as shown in FIG. 6B. In this case, the default widget for the“Start Command” data tag is an indicator widget, so an indicator lightwidget appears in the design canvas of FIG. 6B at the location where theuser dropped the tag, and the indicator widget is automatically assignedthe “Start Command” data tag value for the “Extruder001” machine. Ingeneral, dragging a data tag value from the left-hand “Model” toolboxpanel to the right-hand display design canvas side will result in thevalue being displayed visually with a compatible widget, but the user isable to edit the properties of the default widget to select a differentdisplay type from a list of compatible widgets, such as an indicator,value box, trend diagram, or any other type of visualization element.The user could also assign a different data tag value to the widget byediting the widget's properties. An example of widget behavior when auser attempts to position a widget in the same space on the canvas gridwhere another widget is already positioned will now be described withrespect to FIGS. 7A and 7B.

FIGS. 7A and 7B are block diagrams that illustrate an operationalscenario involving computing system 110 in an exemplary implementation.In FIG. 7A, a user has positioned an indicator widget on the displaycanvas and assigned a “Start Command” tag value to the widget.Subsequently, the user selects a “Gauge” widget from the left-hand“Widgets” toolbox panel. The user has not yet assigned a value to thegauge, so the gauge displays questions marks instead of an actual value.In this example, the user is in the process of dragging the gauge overto the right, as indicated by the arrow appearing in FIG. 7A. As theuser drags the gauge over the existing “Start Command” indicator widgetthat is already positioned on the grid, the indicator widget isconfigured to move out of the way and make space for the gauge widgetthat is colliding with it. Thus, as shown by the arrow appearing in FIG.7B, the “Start Command” indicator widget automatically moves downward toavoid the new gauge widget as the user drags the gauge into the space onthe grid formerly occupied by the “Start Command” indicator. Note thatalthough the “Start Command” indicator widget moves downward in thisexample to make room for the gauge, the existing indicator widget couldalso avoid the overlapping gauge widget by automatically moving in anyother direction in other examples. Also, this avoiding mechanism occurswhenever a user drags one widget over another, whether that widget wasjust dragged from the widget toolbox panel or the user is re-positioningan existing widget inside the display composer canvas. An example ofwidget behavior when a user drags a data item from the “Model” toolboxpanel onto an existing widget will now be described with respect toFIGS. 8A and 8B.

FIGS. 8A and 8B are block diagrams that illustrate an operationalscenario involving computing system 110 in an exemplary implementation.In FIG. 8A, a user has positioned an indicator widget on the displaycanvas and assigned a “Start Command” tag value to the widget.Subsequently, the user has positioned a blank “Gauge” widget from theleft-hand “Widgets” toolbox panel above the “Start Command” indictor onthe canvas grid, which initially does not have a value assigned to it.In order to assign a value, the user selects the “Hydraulic Pressure”data item from the “Tags” section of the “Model” toolbox panel and dragsthe data tag over to the blank gauge widget. Instead of the gauge widgetautomatically moving out of the way of the new data tag when it isdragged onto the gauge widget as discussed above with respect to FIGS.7A and 7B when the user drags a widget over another widget, thisavoiding mechanism is suppressed. In this case, the widget reactsinstead by indicating whether or not the data type of the tag the userdragged onto the widget is compatible with that widget type, such as bychanging its boarder color when a data tag is dragged onto it toindicate compatibility. For example, the widget could display a blueboarder to indicate that the widget is compatible with the data type ofthe tag and is able to visualize this type of tag, or could display ared color to indicate that the widget is incompatible with the data typeof the tag and thus unable to visualize the tag. The widget couldindicate compatibility with a data item that is dragged onto it by othermeans as well, such as by displaying a textual notification to the user.As illustrated in FIG. 8A, the user drags a compatible data tag of“Hydraulic Pressure” onto the blank gauge widget, so the gauge widgetreacts by changing its boarder color to blue to indicate to the userthat the gauge widget can visualize the “Hydraulic Pressure” data value.Conversely, FIG. 8B illustrates the behavior when the user attempts todrag an incompatible “Start Command” data tag onto the gauge widget,which reacts by displaying a red boarder color on the gauge widget and anotification of “Invalid data type” to the user. An example of a userresizing a widget will now be discussed with respect to FIGS. 9A and 9B.

FIGS. 9A and 9B are block diagrams that illustrate an operationalscenario involving computing system 110 in an exemplary implementation.In FIG. 9A, a user has positioned an indicator widget on the displaycanvas and assigned a “Start Command” tag value to the widget. The userhas also selected a “Gauge” widget from the left-hand “Widgets” toolboxpanel and has positioned the gauge on the canvas grid above the “StartCommand” indicator widget. The user has not yet assigned a value to thegauge, so the gauge displays questions marks instead of an actual value.In order to resize the gauge widget, the user can hover a cursor overthe bottom right-hand corner of the widget, which causes a resize arrowicon to be displayed. To facilitate this discussion, a circle surroundsthe resize arrow icon in FIG. 9A to highlight the resize option, butnote that the resize arrow icon is not typically encircled on theapplication's user interface. Once selected, the user can drag thecorner of the gauge widget to resize the widget graphic. In thisexample, the user increases the size of the gauge widget as shown inFIG. 9B, but the user could also decrease the size of the widget bydragging the corner in the opposite direction. As illustrated in FIG.9B, the avoiding mechanism described in FIG. 7B above is also enabled,so that the “Start Command” indicator widget automatically moves downand out of the way of the expanding gauge widget as it is resized. Anexample of a user accessing a context menu to adjust properties of awidget will now be discussed with respect to FIGS. 10A, 10B, and 10C.

FIGS. 10A, 10B, and 10C are block diagrams that illustrate anoperational scenario involving computing system 110 in an exemplaryimplementation. Each widget has a “gear” icon appearing in the top-rightcorner of the widget, as shown encircled on the gauge widget in FIG. 10Afor the purpose of highlighting this feature. When selected, the gearicon brings up a context menu associated with the widget, as shown inFIG. 10B. In this example, the context menu provides the user with theability to select “Properties” of the widget, open a list of “CompatibleWidgets”, add a new gauge widget instance using the “New Gauge” option,or delete the widget. Note that the options shown in the context menu ofFIG. 10B are merely exemplary, and other types of menu options arepossible and within the scope of this disclosure. The effect of the userselecting the “Properties” option from the context menu in FIG. 10B isillustrated in FIG. 10C. An example of configurable properties for the“Gauge 1” widget is shown in FIG. 10C. Each widget type may havedifferent configurable properties, and the user can configure theproperties of each individual widget appearing on a display composercanvas. In the properties window, the user is able to set variousattributes of the gauge. For example, the value for the gauge iscurrently set to receive the data from the model, but the user is ableto change the value to point to a different data source if desired. Theuser can also add a caption for the gauge, such as a description of whatvalue the gauge is displaying. Further, the user can specify theengineering unit for the gauge, and set the gauge range by enteringvalues into the minimum and maximum value boxes. The user can alsochange the style of the gauge, which is set to “Speed meter” in thisexample. The level of decimal precision for the gauge can also be set,along with the behavior when the data displayed by the gauge is outsideof the range, such as an out-of-range color or font color. Of course,the number and type of options that are shown in the properties windowwill differ depending on the type of widget selected and the value beingrepresented by the widget, among other factors. The properties shown forthe gauge in FIG. 10C are merely exemplary. An example of differentdisplay style options for a user's customized display will now bediscussed with respect to FIGS. 11A, 11B, 11C, and 11D.

FIGS. 11A, 11B, 10C, and 11D are block diagrams that illustrate anoperational scenario involving computing system 110 in an exemplaryimplementation. At the bottom of the display composer interface, theuser is presented with an option to adjust display styles for the entiredisplay. The display styles dialog menu is shown encircled in FIG. 11Afor the purpose of highlighting this feature. When selected, the displaystyles option brings up the “Display Styles” menu for editing thedisplay style for the entire display, as shown in FIG. 11B. In thisexample, the Display Styles” menu provides the user with the ability tochange the “Theme palette”, alter the “Display background color”, andchange the “Cell background color” of a particular widget. Note that thedisplay options shown in the “Display Styles” menu on FIG. 11B aremerely exemplary, and other types of menu options are possible andwithin the scope of this disclosure. In FIG. 11C, the user has activatedthe color selector for the “Display background color” option. The colorselector is broken up into two sections, labeled “Theme colors” and“Standard colors”. The theme colors depend on the selected themepalette. When the theme color palette is changed, all display items thatare using theme colors are changed. The standard colors are static, andthus any display items that are assigned a color from the “Standardcolors” selection are unaffected if the theme palette is changed. Inthis example, the user changes the display background color to orangeand the cell background color to red. An example of a “styled” displayis illustrated in FIG. 11D, which reflects the user's changes of tobackground color for the display canvas and a different background colorfor the widgets. In this manner, the user is able to create fullycustomized displays and dashboards for viewing dynamic data associatedwith industrial automation operations in a style and layout as desiredby the user.

Referring now to FIG. 12, a block diagram that illustrates computingsystem 1200 in an exemplary implementation is shown. Computing system1200 provides an example of computing system 110, although system 110could use alternative configurations. Computing system 1200 includesprocessing system 1201, storage system 1203, software 1205,communication interface 1207, and user interface 1209. User interface1209 comprises display system 1208. Software 1205 includes application1206 which itself includes visualization process 200. However, in someimplementations, visualization process 200 may optionally be implementedseparately from application 1206, as shown by the dotted line around thegraphical representation of process 200 appearing outside of application1206.

Computing system 1200 may be representative of any computing apparatus,system, or systems on which application 1206 and visualization process200 or variations thereof may be suitably implemented. Examples ofcomputing system 1200 include mobile computing devices, such as cellphones, tablet computers, laptop computers, notebook computers, andgaming devices, as well as any other type of mobile computing devicesand any combination or variation thereof. Note that the features andfunctionality of computing system 1200 may apply as well to desktopcomputers, server computers, and virtual machines, as well as any othertype of computing system, variation, or combination thereof.

Computing system 1200 includes processing system 1201, storage system1203, software 1205, communication interface 1207, and user interface1209. Processing system 1201 is operatively coupled with storage system1203, communication interface 1207, and user interface 1209. Processingsystem 1201 loads and executes software 1205 from storage system 1203.When executed by computing system 1200 in general, and processing system1201 in particular, software 1205 directs computing system 1200 tooperate as described herein for a visualization process or variationsthereof. Computing system 1200 may optionally include additionaldevices, features, or functionality not discussed herein for purposes ofbrevity.

Referring still to FIG. 12, processing system 1201 may comprise amicroprocessor and other circuitry that retrieves and executes software1205 from storage system 1203. Processing system 1201 may be implementedwithin a single processing device but may also be distributed acrossmultiple processing devices or sub-systems that cooperate in executingprogram instructions. Examples of processing system 1201 include generalpurpose central processing units, application specific processors, andlogic devices, as well as any other type of processing device,combinations, or variations thereof.

Storage system 1203 may comprise any computer readable media or storagemedia readable by processing system 1201 and capable of storing software1205. Storage system 1203 may include volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information, such as computer readableinstructions, data structures, program modules, or other data. Storagesystem 1203 may be implemented as a single storage device but may alsobe implemented across multiple storage devices or sub-systems co-locatedor distributed relative to each other. Storage system 1203 may compriseadditional elements, such as a controller, capable of communicating withprocessing system 1201. Examples of storage media include random accessmemory, read only memory, magnetic disks, optical disks, flash memory,virtual memory and non-virtual memory, magnetic cassettes, magnetictape, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store the desired information and thatmay be accessed by an instruction execution system, as well as anycombination or variation thereof, or any other type of storage media. Inno case is the storage media a propagated signal.

In operation, in conjunction with user interface 1209, processing system1201 loads and executes portions of software 1205, such as application1206 and/or visualization process 200, to render a graphical userinterface for application 1206 for display by display system 1208 ofuser interface 1209. Software 1205 may be implemented in programinstructions and among other functions may, when executed by computingsystem 1200 in general or processing system 1201 in particular, directcomputing system 1200 or processing system 1201 to present a displaycomposer interface that enables a user to design a customized displaylayout for the application associated with the industrial automationenvironment. Software 1205 may further direct computing system 1200 orprocessing system 1201 to receive data display instructions comprising auser selection of at least one data item associated with an operation ofa machine in the industrial automation environment, and receive positioninformation that identifies where to display the at least one data item.Software 1205 may also direct computing system 1200 or processing system1201 to process the data display instructions and the positioninformation to generate the customized display layout for theapplication. In addition, software 1205 may direct computing system 1200or processing system 1201 to, based on the customized display layout,render a graphical user interface to the application having the at leastone data item positioned according to the position information.

Software 1205 may include additional processes, programs, or components,such as operating system software or other application software.Examples of operating systems include Windows®, iOS®, and Android®, aswell as any other suitable operating system. Software 1205 may alsocomprise firmware or some other form of machine-readable processinginstructions executable by processing system 1201.

In general, software 1205 may, when loaded into processing system 1201and executed, transform computing system 1200 overall from ageneral-purpose computing system into a special-purpose computing systemcustomized to facilitate visualization of an application as describedherein for each implementation. For example, encoding software 1205 onstorage system 1203 may transform the physical structure of storagesystem 1203. The specific transformation of the physical structure maydepend on various factors in different implementations of thisdescription. Examples of such factors may include, but are not limitedto the technology used to implement the storage media of storage system1203 and whether the computer-storage media are characterized as primaryor secondary storage.

In some examples, if the computer-storage media are implemented assemiconductor-based memory, software 1205 may transform the physicalstate of the semiconductor memory when the program is encoded therein.For example, software 1205 may transform the state of transistors,capacitors, or other discrete circuit elements constituting thesemiconductor memory. A similar transformation may occur with respect tomagnetic or optical media. Other transformations of physical media arepossible without departing from the scope of the present description,with the foregoing examples provided only to facilitate this discussion.

It should be understood that computing system 1200 is generally intendedto represent a computing system with which software 1205 is deployed andexecuted in order to implement application 1206 and/or visualizationprocess 200 (and variations thereof). However, computing system 1200 mayalso represent any computing system on which software 1205 may be stagedand from where software 1205 may be distributed, transported,downloaded, or otherwise provided to yet another computing system fordeployment and execution, or yet additional distribution. For example,computing system 1200 could be configured to deploy software 1205 overthe internet to one or more client computing systems for executionthereon, such as in a cloud-based deployment scenario.

Communication interface 1207 may include communication connections anddevices that allow for communication between computing system 1200 andother computing systems (not shown) or services, over a communicationnetwork 1211 or collection of networks. In some implementations,communication interface 1207 receives dynamic data 1221 overcommunication network 1211. Examples of connections and devices thattogether allow for inter-system communication may include networkinterface cards, antennas, power amplifiers, RF circuitry, transceivers,and other communication circuitry. The aforementioned network,connections, and devices are well known and need not be discussed atlength here.

User interface 1209 may include a voice input device, a touch inputdevice for receiving a gesture from a user, a motion input device fordetecting non-touch gestures and other motions by a user, and othercomparable input devices and associated processing elements capable ofreceiving user input from a user. Output devices such as a display,speakers, haptic devices, and other types of output devices may also beincluded in user interface 1209. In this example, user interface 1209includes display system 1208, which itself may be part of a touch screenthat also accepts user inputs via touches on its surface. Theaforementioned user input devices are well known in the art and need notbe discussed at length here. User interface 1209 may also includeassociated user interface software executable by processing system 1201in support of the various user input and output devices discussed above.Separately or in conjunction with each other and other hardware andsoftware elements, the user interface software and devices may provide agraphical user interface, a natural user interface, or any other kind ofuser interface.

The functional block diagrams, operational sequences, and flow diagramsprovided in the Figures are representative of exemplary architectures,environments, and methodologies for performing novel aspects of thedisclosure. While, for purposes of simplicity of explanation, methodsincluded herein may be in the form of a functional diagram, operationalsequence, or flow diagram, and may be described as a series of acts, itis to be understood and appreciated that the methods are not limited bythe order of acts, as some acts may, in accordance therewith, occur in adifferent order and/or concurrently with other acts from that shown anddescribed herein. For example, those skilled in the art will understandand appreciate that a method could alternatively be represented as aseries of interrelated states or events, such as in a state diagram.Moreover, not all acts illustrated in a methodology may be required fora novel implementation.

The above description and associated drawings teach the best mode of theinvention. The following claims specify the scope of the invention. Someaspects of the best mode may not fall within the scope of the inventionas specified by the claims. Also, while the preceding discussiondescribes embodiments employed specifically in conjunction with themonitoring and analysis of industrial processes, other applications,such as the mathematical modeling or monitoring of any man-made ornaturally-existing system, may benefit from use of the conceptsdiscussed above. Further, those skilled in the art will appreciate thatthe features described above can be combined in various ways to formmultiple variations of the invention. As a result, the invention is notlimited to the specific embodiments described above, but only by thefollowing claims and their equivalents.

What is claimed is:
 1. A method of operating a computing system to facilitate visualization of an application associated with an industrial automation environment, the method comprising: presenting a display composer interface comprising a visualization layout editor for designing a customized display layout for the application associated with the industrial automation environment; receiving data display instructions comprising an indication of at least one data item associated with an operation of a machine in the industrial automation environment and a data display type for displaying the at least one data item, wherein a default data display type compatible with the at least one data item is automatically created for the at least one data item and the at least one data item is automatically assigned to the default data display type which serves as the data display type for displaying the at least one data item; receiving position information that identifies where to display the at least one data item; processing the data display instructions and the position information to generate the customized display layout for the application; and rendering a graphical user interface to the application having the at least one data item displayed according to the data display type and positioned according to the position information.
 2. The method of claim 1 wherein receiving the data display instructions comprises receiving display size instructions for the at least one data item.
 3. The method of claim 1 wherein processing the data display instructions and the position information to generate the customized display layout for the application comprises altering the position information that identifies where to display the at least one data item to avoid overlapping with other data items that were previously positioned in the customized display layout.
 4. The method of claim 1 wherein receiving the data display instructions comprising the user selection of the at least one data item and the data display type for displaying the at least one data item comprises receiving instructions to edit properties of the default data display type to select a different data display type from a list of compatible data display types to serve as the data display type for displaying the at least one data item.
 5. The method of claim 1 wherein the data display type comprises an indicator.
 6. The method of claim 1 wherein the data display type comprises a gauge.
 7. The method of claim 1 wherein the data display type comprises a value box.
 8. One or more computer-readable storage media having program instructions stored thereon to facilitate visualization of an application associated with an industrial automation environment, wherein the program instructions, when executed by a computing system, direct the computing system to at least: present a display composer interface comprising a visualization layout editor for designing a customized display layout for the application associated with the industrial automation environment; receive data display instructions comprising an indication of at least one data item associated with an operation of a machine in the industrial automation environment and a data display type for displaying the at least one data item, wherein a default data display type compatible with the at least one data item is automatically created for the at least one data item and the at least one data item is automatically assigned to the default data display type which serves as the data display type for displaying the at least one data item; receive position information that identifies where to display the at least one data item; process the data display instructions and the position information to generate the customized display layout for the application; and based on the customized display layout, render a graphical user interface to the application having the at least one data item displayed according to the data display type and positioned according to the position information.
 9. The one or more computer-readable storage media of claim 8 wherein the program instructions direct the computing system to receive the data display instructions by directing the computing system to receive display size instructions for the at least one data item.
 10. The one or more computer-readable storage media of claim 8 wherein the program instructions direct the computing system to process the data display instructions and the position information to generate the customized display layout for the application by directing the computing system to alter the position information that identifies where to display the at least one data item to avoid overlapping with other data items that were previously positioned in the customized display layout.
 11. The one or more computer-readable storage media of claim 8 wherein the program instructions direct the computing system to receive the data display instructions comprising the user selection of the at least one data item and the data display type for displaying the at least one data item by directing the computing system to receive instructions to edit properties of the default data display type to select a different data display type from a list of compatible data display types to serve as the data display type for displaying the at least one data item.
 12. The one or more computer-readable storage media of claim 8 wherein the data display type comprises an indicator.
 13. The one or more computer-readable storage media of claim 8 wherein the data display type comprises a gauge.
 14. The one or more computer-readable storage media of claim 8 wherein the data display type comprises a value box.
 15. An apparatus to facilitate visualization of an application associated with an industrial automation environment, the apparatus comprising: one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media that, when executed by a processing system, direct the processing system to at least: present a display composer interface that provides a visualization layout editor for editing a customized display layout for the application associated with the industrial automation environment; receive data display instructions comprising a user selection of at least one data item associated with an operation of a machine in the industrial automation environment and a data display type for displaying the at least one data item, wherein a default data display type compatible with the at least one data item is automatically created for the at least one data item identified in the user selection and the at least one data item is automatically assigned to the default data display type which serves as the data display type for displaying the at least one data item; receive position information that identifies where to display the at least one data item; process the data display instructions and the position information to generate the customized display layout for the application; and produce a graphical user interface to the application having the at least one data item displayed according to the data display type and positioned according to the position information.
 16. The apparatus of claim 15 wherein the program instructions direct the processing system to receive the data display instructions by directing the processing system to receive display size instructions for the at least one data item.
 17. The apparatus of claim 15 wherein the program instructions direct the processing system to process the data display instructions and the position information to generate the customized display layout for the application by directing the processing system to alter the position information that identifies where to display the at least one data item to avoid overlapping with other data items that were previously positioned in the customized display layout.
 18. The apparatus of claim 15 wherein the program instructions direct the processing system to receive the data display instructions comprising the user selection of the at least one data item and the data display type for displaying the at least one data item by directing the processing system to receive instructions to edit properties of the default data display type to select a different data display type from a list of compatible data display types to serve as the data display type for displaying the at least one data item.
 19. The apparatus of claim 15 wherein the data display type comprises an indicator.
 20. The apparatus of claim 15 wherein the data display type comprises a gauge. 